Learning visuo-motor coordination for pointing without depth calculation

Pointing refers to orienting a hand, arm, head or body towards an object and is possible without calculating the object’s depth and 3D position. We show that pointing can be learned as holistic direct mapping from an object’s pixel coordinates in the visual field to joint angles, which define pose and orientation of a human or robot. To this aim, we record real world and noisy training images together with corresponding robot pointing postures for the humanoid robot iCub. We then learn and comparatively evaluate pointing with an multi-layer perceptron, an extreme learning machine and a reservoir network, but also demonstrate that learning fails at reconstructing the depth of trained objects.